class: center, middle, inverse, title-slide # Getting the most out of R ## Physalia Courses ### Luis D. Verde Arregoitia ### February 2022 --- <style type="text/css"> .dk-section-title { box-shadow: inset 0 0 0 1000px rgba(0,0,0,0.5); } .dk-section-title h1 { color: #f3f3f3; font-size: 100px; line-height: 1; } </style> # Day 2 Unfinished business .f3[ Organizing data Principles of rectangular data Tidy Data Data Sharing Tools for data rectangling (tidyverse-oriented) ] --- ## Unfinished business .f3[ Disambiguate `here` - see slides for Day 1 Loading data bundled with packages (and objects of type `promise`) - we will do this for the exercises ] --- class: dk-section-title background-image:url("imgs/bsas.jpg") background-size: cover # Data organization --- # Well-organized data - Easier to import and manipulate - Less prone to (non-replicable) errors - omission - alteration - duplication .f3[ Good data organization helps us when we produce data, and also to identify and solve problems in existing data. ] --- ## Data Organization best practices .f3[ - Consistency in names, layouts, abbreviations, units, etc. - Good names - Make it rectangular ] .pull-left[ Openscapes (2020) [Tidy data illustrated](https://www.openscapes.org/blog/2020/10/12/tidy-data/) .bg-yellow.b[ Broman and Woo (2018) [Organizing data in spreadsheets](https://doi.org/10.1080/00031305.2017.1375989)] ] .pull-right[ Verde Arregoita et al. (2018) [Good practices for sharing analysis-ready data...](https://doi.org/10.4404/hystrix-00133-2018) ] --- ## Consistency <img src="data:image/png;base64,#imgs/consistency.png" width="74%" /> .tr[ Find the inconsistencies ] --- ## Data rectangles .fl.w-30[ <img src="data:image/png;base64,#imgs/rectangle1.png" width="76%" /> ] .fl.w-70[ Objects with rows and columns, in which: - Each row has the same number of columns - Each column has the same number of rows _Vectors and factors of the same length = all columns must have the same 'height'_ Equal-height columns give a rectangular shape to the data frame ] --- .fl.w-50[ <img src="data:image/png;base64,#imgs/cashflow.png" width="50%" /> Davis Vaughan (2018) [Tidying Excel cash flow spreadsheets]( https://blog.davisvaughan.com/2018/02/16/tidying-excel-cash-flow-spreadsheets-in-r/) ] .f3[ rectangular data ~ a range of cells in a spreadsheet ] <img src="data:image/png;base64,#imgs/rangespr.png" width="50%" /> --- <blockquote class="twitter-tweet tw-align-center" data-conversation="none"><p lang="en" dir="ltr">but sometimes such data is “rectangular” in the same sense that this cat is “bowl-shaped” 😂 <a href="https://t.co/ZmxvgcV57d">pic.twitter.com/ZmxvgcV57d</a></p>&mdash; Jenny Bryan (@JennyBryan) <a href="https://twitter.com/JennyBryan/status/1126582138344595456?ref_src=twsrc%5Etfw">May 9, 2019</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script> --- class: middle <img src="data:image/png;base64,#imgs/transposed.png" width="80%" style="display: block; margin: auto;" /> --- class: center, middle, dk-section-title background-image:url("imgs/peru.jpg") background-size: cover # Tidy Data and Data Rectangling --- ## Ready for immediate use? - Subset, group, transform, sort, visualize, model, share ## Not ready? - Restructure for easier and more effective handling --- ## Tidy data An accessible reformulation of existing principles and definitions from statistics and computer science, to ultimately help us structure our data for maximum usability. - Arranged in a 2D data matrix with rows and columns - Rows correspond to observations - Columns correspond to variables - One value per cell .tr[ Wickham (2014) [Tidy Data](https://www.jstatsoft.org/article/view/v059i10/) - Journal of Statistical Software ] --- <img src="data:image/png;base64,#imgs/varscols.png" width="85%" style="display: block; margin: auto;" /> --- <img src="data:image/png;base64,#imgs/tidydata.png" width="70%" style="display: block; margin: auto;" /> --- ## Why tidy? .fl.w-40[ Easy to reshape into other structures after cleaning or transforming ] <img src="data:image/png;base64,#imgs/pivotingintro.gif" width="42%" /> --- ## Why tidy? .f3[Row-wise operations] Each value in an observation will be paired with other values that appear in the same row <img src="data:image/png;base64,#imgs/discounts.png" width="60%" style="display: block; margin: auto;" /> --- class: middle .fl.w-40[ <img src="data:image/png;base64,#imgs/tidyvarsmap.png" width="85%" style="display: block; margin: auto auto auto 0;" /> ] .fl.w-60[ .f3[ Variables feed directly into model specifications and plotting parameters. ]] --- ## Why not .f3[ Tidy data -> more repetitive, often takes up more space than other more condensed representations Not ideal for data entry, or for tables that will be embedded in a text document ] --- class: center, middle, dk-section-title background-image:url("imgs/trote.jpg") background-size: cover # Data organization quick tips --- ### Grouping information goes in its own variable. Use indicator variables instead of encoding information into an existing value. <img src="data:image/png;base64,#imgs/indgroups.png" width="90%" style="display: block; margin: auto;" /> --- ## NA and Empty Values Not recorded, not applicable, lost in transcription, etc. .center.black.f2[Missing data are still data] In R we use .red[`NA`], a special logical constant that denotes missing values --- class: middle - Code .red.b[`NA`] values consistently (if done consistently, blank cells are OK) - Avoid ambiguity > Are data known to be missing, or was a value intentionally left blank? - Avoid empty cells as padding to align values, or for aesthetic or layout purposes. - Avoid .red.b[`NA`] values in column headers --- <img src="data:image/png;base64,#imgs/NAblank.png" width="55%" style="display: block; margin: auto;" /> --- ## Make reasonable headers (column names) uninformative names with inconsistent naming schemes <div id="rokmbsygbc" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #rokmbsygbc .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #FFFFFF; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #rokmbsygbc .gt_heading { background-color: #FFFFFF; text-align: center; border-bottom-color: #FFFFFF; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #rokmbsygbc .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #FFFFFF; border-bottom-width: 0; } #rokmbsygbc .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #FFFFFF; border-top-width: 0; } #rokmbsygbc .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #rokmbsygbc .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #rokmbsygbc .gt_col_heading { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #rokmbsygbc .gt_column_spanner_outer { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #rokmbsygbc .gt_column_spanner_outer:first-child { padding-left: 0; } #rokmbsygbc .gt_column_spanner_outer:last-child { padding-right: 0; } #rokmbsygbc .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #rokmbsygbc .gt_group_heading { padding: 8px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #rokmbsygbc .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #rokmbsygbc .gt_from_md > :first-child { margin-top: 0; } #rokmbsygbc .gt_from_md > :last-child { margin-bottom: 0; } #rokmbsygbc .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #rokmbsygbc .gt_stub { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #rokmbsygbc .gt_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #rokmbsygbc .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #rokmbsygbc .gt_grand_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #rokmbsygbc .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #rokmbsygbc .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #rokmbsygbc .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #rokmbsygbc .gt_footnotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #rokmbsygbc .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #rokmbsygbc .gt_sourcenotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #rokmbsygbc .gt_sourcenote { font-size: 90%; padding: 4px; } #rokmbsygbc .gt_left { text-align: left; } #rokmbsygbc .gt_center { text-align: center; } #rokmbsygbc .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #rokmbsygbc .gt_font_normal { font-weight: normal; } #rokmbsygbc .gt_font_bold { font-weight: bold; } #rokmbsygbc .gt_font_italic { font-style: italic; } #rokmbsygbc .gt_super { font-size: 65%; } #rokmbsygbc .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">X</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">X1</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">X2</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">min_Score</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">AVERAGE.SCORE</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 25px;">UMN</td> <td class="gt_row gt_left" style="font-size: 25px;">EAST</td> <td class="gt_row gt_left" style="font-size: 25px;">A</td> <td class="gt_row gt_right" style="font-size: 25px;">7.7</td> <td class="gt_row gt_right" style="font-size: 25px;">7.701</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">UV</td> <td class="gt_row gt_left" style="font-size: 25px;">WEST</td> <td class="gt_row gt_left" style="font-size: 25px;">B</td> <td class="gt_row gt_right" style="font-size: 25px;">8.9</td> <td class="gt_row gt_right" style="font-size: 25px;">8.890</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">UNLV</td> <td class="gt_row gt_left" style="font-size: 25px;">EAST</td> <td class="gt_row gt_left" style="font-size: 25px;">C</td> <td class="gt_row gt_right" style="font-size: 25px;">9.2</td> <td class="gt_row gt_right" style="font-size: 25px;">9.199</td></tr> </tbody> </table> </div> - More difficult to remember and specify - Do not sort well --- .f3[Much better] <div id="fwxljkmmqp" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #fwxljkmmqp .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #FFFFFF; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #fwxljkmmqp .gt_heading { background-color: #FFFFFF; text-align: center; border-bottom-color: #FFFFFF; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #fwxljkmmqp .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #FFFFFF; border-bottom-width: 0; } #fwxljkmmqp .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #FFFFFF; border-top-width: 0; } #fwxljkmmqp .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #fwxljkmmqp .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #fwxljkmmqp .gt_col_heading { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #fwxljkmmqp .gt_column_spanner_outer { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #fwxljkmmqp .gt_column_spanner_outer:first-child { padding-left: 0; } #fwxljkmmqp .gt_column_spanner_outer:last-child { padding-right: 0; } #fwxljkmmqp .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #fwxljkmmqp .gt_group_heading { padding: 8px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #fwxljkmmqp .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #fwxljkmmqp .gt_from_md > :first-child { margin-top: 0; } #fwxljkmmqp .gt_from_md > :last-child { margin-bottom: 0; } #fwxljkmmqp .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #fwxljkmmqp .gt_stub { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #fwxljkmmqp .gt_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #fwxljkmmqp .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #fwxljkmmqp .gt_grand_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #fwxljkmmqp .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #fwxljkmmqp .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #fwxljkmmqp .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #fwxljkmmqp .gt_footnotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #fwxljkmmqp .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #fwxljkmmqp .gt_sourcenotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #fwxljkmmqp .gt_sourcenote { font-size: 90%; padding: 4px; } #fwxljkmmqp .gt_left { text-align: left; } #fwxljkmmqp .gt_center { text-align: center; } #fwxljkmmqp .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #fwxljkmmqp .gt_font_normal { font-weight: normal; } #fwxljkmmqp .gt_font_bold { font-weight: bold; } #fwxljkmmqp .gt_font_italic { font-style: italic; } #fwxljkmmqp .gt_super { font-size: 65%; } #fwxljkmmqp .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">university</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">coast</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">division</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">min_score</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">avg_score</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 25px;">UMN</td> <td class="gt_row gt_left" style="font-size: 25px;">EAST</td> <td class="gt_row gt_left" style="font-size: 25px;">A</td> <td class="gt_row gt_right" style="font-size: 25px;">7.7</td> <td class="gt_row gt_right" style="font-size: 25px;">7.701</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">UV</td> <td class="gt_row gt_left" style="font-size: 25px;">WEST</td> <td class="gt_row gt_left" style="font-size: 25px;">B</td> <td class="gt_row gt_right" style="font-size: 25px;">8.9</td> <td class="gt_row gt_right" style="font-size: 25px;">8.890</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">UNLV</td> <td class="gt_row gt_left" style="font-size: 25px;">EAST</td> <td class="gt_row gt_left" style="font-size: 25px;">C</td> <td class="gt_row gt_right" style="font-size: 25px;">9.2</td> <td class="gt_row gt_right" style="font-size: 25px;">9.199</td></tr> </tbody> </table> </div> --- class: middle ### Names broken across rows .pull-left[ <div id="ssrjbmovpu" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #ssrjbmovpu .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #FFFFFF; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #ssrjbmovpu .gt_heading { background-color: #FFFFFF; text-align: center; border-bottom-color: #FFFFFF; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #ssrjbmovpu .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #FFFFFF; border-bottom-width: 0; } #ssrjbmovpu .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #FFFFFF; border-top-width: 0; } #ssrjbmovpu .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #ssrjbmovpu .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #ssrjbmovpu .gt_col_heading { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #ssrjbmovpu .gt_column_spanner_outer { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #ssrjbmovpu .gt_column_spanner_outer:first-child { padding-left: 0; } #ssrjbmovpu .gt_column_spanner_outer:last-child { padding-right: 0; } #ssrjbmovpu .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #ssrjbmovpu .gt_group_heading { padding: 8px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #ssrjbmovpu .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #ssrjbmovpu .gt_from_md > :first-child { margin-top: 0; } #ssrjbmovpu .gt_from_md > :last-child { margin-bottom: 0; } #ssrjbmovpu .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #ssrjbmovpu .gt_stub { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #ssrjbmovpu .gt_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #ssrjbmovpu .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #ssrjbmovpu .gt_grand_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #ssrjbmovpu .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #ssrjbmovpu .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #ssrjbmovpu .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #ssrjbmovpu .gt_footnotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #ssrjbmovpu .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #ssrjbmovpu .gt_sourcenotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #ssrjbmovpu .gt_sourcenote { font-size: 90%; padding: 4px; } #ssrjbmovpu .gt_left { text-align: left; } #ssrjbmovpu .gt_center { text-align: center; } #ssrjbmovpu .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #ssrjbmovpu .gt_font_normal { font-weight: normal; } #ssrjbmovpu .gt_font_bold { font-weight: bold; } #ssrjbmovpu .gt_font_italic { font-style: italic; } #ssrjbmovpu .gt_super { font-size: 65%; } #ssrjbmovpu .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">X</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">X1</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">X2</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">min</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">AVERAGE</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 24px;">NA</td> <td class="gt_row gt_left" style="font-size: 24px;">NA</td> <td class="gt_row gt_left" style="font-size: 24px;">NA</td> <td class="gt_row gt_left" style="font-size: 24px;">Score</td> <td class="gt_row gt_left" style="font-size: 24px;">SCORE</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">UMN</td> <td class="gt_row gt_left" style="font-size: 24px;">EAST</td> <td class="gt_row gt_left" style="font-size: 24px;">A</td> <td class="gt_row gt_left" style="font-size: 24px;">7.7</td> <td class="gt_row gt_left" style="font-size: 24px;">7.701</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">UV</td> <td class="gt_row gt_left" style="font-size: 24px;">WEST</td> <td class="gt_row gt_left" style="font-size: 24px;">B</td> <td class="gt_row gt_left" style="font-size: 24px;">8.9</td> <td class="gt_row gt_left" style="font-size: 24px;">8.89</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">UNLV</td> <td class="gt_row gt_left" style="font-size: 24px;">EAST</td> <td class="gt_row gt_left" style="font-size: 24px;">C</td> <td class="gt_row gt_left" style="font-size: 24px;">9.2</td> <td class="gt_row gt_left" style="font-size: 24px;">9.199</td></tr> </tbody> </table> </div> ] .pull-right.f3[ Variable names appear in >1 rows Header fragments mixed with data Separators become implicit `NAs` introduced ] --- class: center, middle, dk-section-title background-image:url("imgs/brigalow.jpg") background-size: cover # Data Sharing Best Practices --- ## Data can be intuitive, easy to read AND machine-readable if shared adequately .f3[ ↘️ Reduce the need for complex/proprietary software or advanced programming skills for inclusion in future analyses. ↘ Minimize the amount of interpretation that others (or future you) will need to make to use the data. ] --- ## File formats .f3[ - Avoid PDF as a data format - The simpler the better (prefer delimited text files such as CSVs) - Share data-only files, keep calculations and results elsewhere ] --- class: dk-section-title background-image:url("imgs/trote.jpg") background-size: cover # Tools for data rectangling --- class: inverse ## Project setup .f3.white[ - Create a new RStudio Project for this course - Name the project "`gtmor`" and save it somewhere accessible (e.g. the Desktop) - Create the following folders in your working directory: - R - Out - Data ] --- class: inverse ## Project setup (continued) .f3.white[ - Make sure you have `dplyr` installed - Install the '`openintro`' package from CRAN `install.packages("openintro")` - Get a drink or some snacks ] --- class: center, middle, dk-section-title background-image:url("imgs/scl.jpg") background-size: cover # Data manipulation tools --- ### Tools for preparing, transforming, and restructuring data .fl.w-40[ .tc.f3[ 📦 .b[`dplyr`] 📦 .b[`tidyr`] ]] .fl.w-60[ Flexible, but developed around built-in features that support a particular way of doing things ] <br/><br/><br/><br/><br/><br/> - Hard to use incorrectly by accident - Less code to achieve a goal - Shared grammar and input/output data structures - Tasks split up across functions and packages - Imports pipes --- ## Useful 📦 .b.red[`dplyr`] functions - Individual functions for the most common operations - Each function does one only thing, but does it well - Intuitive, user-friendly functions for common data manipulation tasks: - Subset columns - Subset rows - Create or modify columns --- .panelset[ .panel[.panel-name[sneakers] <div id="qmwjesecgs" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #qmwjesecgs .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #FFFFFF; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #qmwjesecgs .gt_heading { background-color: #FFFFFF; text-align: center; border-bottom-color: #FFFFFF; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #qmwjesecgs .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #FFFFFF; border-bottom-width: 0; } #qmwjesecgs .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #FFFFFF; border-top-width: 0; } #qmwjesecgs .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #qmwjesecgs .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #qmwjesecgs .gt_col_heading { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #qmwjesecgs .gt_column_spanner_outer { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #qmwjesecgs .gt_column_spanner_outer:first-child { padding-left: 0; } #qmwjesecgs .gt_column_spanner_outer:last-child { padding-right: 0; } #qmwjesecgs .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #qmwjesecgs .gt_group_heading { padding: 8px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #qmwjesecgs .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #qmwjesecgs .gt_from_md > :first-child { margin-top: 0; } #qmwjesecgs .gt_from_md > :last-child { margin-bottom: 0; } #qmwjesecgs .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #qmwjesecgs .gt_stub { color: #333333; background-color: #FFFFFF; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #qmwjesecgs .gt_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #qmwjesecgs .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #qmwjesecgs .gt_grand_summary_row { color: #333333; background-color: #FFFFFF; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #qmwjesecgs .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #qmwjesecgs .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #qmwjesecgs .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #qmwjesecgs .gt_footnotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #qmwjesecgs .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #qmwjesecgs .gt_sourcenotes { color: #333333; background-color: #FFFFFF; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #qmwjesecgs .gt_sourcenote { font-size: 90%; padding: 4px; } #qmwjesecgs .gt_left { text-align: left; } #qmwjesecgs .gt_center { text-align: center; } #qmwjesecgs .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #qmwjesecgs .gt_font_normal { font-weight: normal; } #qmwjesecgs .gt_font_bold { font-weight: bold; } #qmwjesecgs .gt_font_italic { font-style: italic; } #qmwjesecgs .gt_super { font-size: 65%; } #qmwjesecgs .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 21px;">Name</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 21px;">Sport</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 21px;">Price</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 21px;">Rating</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 21px;">Boost 3000</td> <td class="gt_row gt_left" style="font-size: 21px;">Running</td> <td class="gt_row gt_right" style="font-size: 21px;">99</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">HiSpeed</td> <td class="gt_row gt_left" style="font-size: 21px;">Running</td> <td class="gt_row gt_right" style="font-size: 21px;">44</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Super2</td> <td class="gt_row gt_left" style="font-size: 21px;">Trail</td> <td class="gt_row gt_right" style="font-size: 21px;">100</td> <td class="gt_row gt_right" style="font-size: 21px;">5</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Goal</td> <td class="gt_row gt_left" style="font-size: 21px;">Football</td> <td class="gt_row gt_right" style="font-size: 21px;">119</td> <td class="gt_row gt_right" style="font-size: 21px;">3</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Goal2.0</td> <td class="gt_row gt_left" style="font-size: 21px;">Football</td> <td class="gt_row gt_right" style="font-size: 21px;">79</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Airball Max</td> <td class="gt_row gt_left" style="font-size: 21px;">Basketball</td> <td class="gt_row gt_right" style="font-size: 21px;">49</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Glide Best</td> <td class="gt_row gt_left" style="font-size: 21px;">Running</td> <td class="gt_row gt_right" style="font-size: 21px;">55</td> <td class="gt_row gt_right" style="font-size: 21px;">5</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Outdoor King</td> <td class="gt_row gt_left" style="font-size: 21px;">Trail</td> <td class="gt_row gt_right" style="font-size: 21px;">160</td> <td class="gt_row gt_right" style="font-size: 21px;">5</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">G90x</td> <td class="gt_row gt_left" style="font-size: 21px;">Football</td> <td class="gt_row gt_right" style="font-size: 21px;">120</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Smith Pro</td> <td class="gt_row gt_left" style="font-size: 21px;">Running</td> <td class="gt_row gt_right" style="font-size: 21px;">89</td> <td class="gt_row gt_right" style="font-size: 21px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">Travel Low</td> <td class="gt_row gt_left" style="font-size: 21px;">Basketball</td> <td class="gt_row gt_right" style="font-size: 21px;">89</td> <td class="gt_row gt_right" style="font-size: 21px;">3</td></tr> </tbody> </table> </div> ] .panel[.panel-name[Data setup] ```r sneakers <- tibble::tribble( ~Name, ~Sport, ~Price, ~Rating, "Boost 3000", "Running", 99L, 4L, "HiSpeed", "Running", 44L, 4L, "Super2", "Trail", 100L, 5L, "Goal", "Football", 119L, 3L, "Goal2.0", "Football", 79L, 4L, "Airball Max", "Basketball", 49L, 4L, "Glide Best", "Running", 55L, 5L, "Outdoor King", "Trail", 160L, 5L, "G90x", "Football", 120L, 4L, "Smith Pro", "Running", 89L, 4L, "Travel Low", "Basketball", 89L, 3L) ``` ] ] --- .left-column[ .f3.b[`select()`] .f3[Subset variables] - by name - by position - by type ] .right-column[ ```r sneakers %>% select(Name, Sport) %>% head(4) ``` ``` ## # A tibble: 4 × 2 ## Name Sport ## <chr> <chr> ## 1 Boost 3000 Running ## 2 HiSpeed Running ## 3 Super2 Trail ## 4 Goal Football ``` ] --- .left-column[ .f3.b[`mutate()`] .f3[Create and modify columns] - create new - overwrite ] .right-column[ ```r sneakers %>% select(Name, Rating) %>% mutate(Rating= Rating*2) %>% head(5) ``` ``` ## # A tibble: 5 × 2 ## Name Rating ## <chr> <dbl> ## 1 Boost 3000 8 ## 2 HiSpeed 8 ## 3 Super2 10 ## 4 Goal 6 ## 5 Goal2.0 8 ``` ] --- .left-column[ .f3.b[`filter()`] .f3[subset rows] - keep rows that meet a condition ] .right-column[ ```r sneakers %>% select(Name, Sport, Rating) %>% filter(Rating == 5) ``` ``` ## # A tibble: 3 × 3 ## Name Sport Rating ## <chr> <chr> <int> ## 1 Super2 Trail 5 ## 2 Glide Best Running 5 ## 3 Outdoor King Trail 5 ``` ] --- ## Pre-exercise warmup - Loading package data .f2.b.blue[`data()`] .f2[loads or lists available datasets] <br/> .f3.b.blue[`data()`] .f3[with no arguments shows every dataset currently available] --- ## Pre-exercise warmup - Loading package data .f3[We can also specify which package we want to examine for bundled data: .center[`data(package="ggplot2")`] .center[`data(package="datasets")`] .center[`data(package="tidyr")`] ] --- ## Pre-exercise warmup - Loading package data .f3[We can use `data()` to load a particular dataset from a package (quoted or unquoted both work)] > Load the relevant package first! ```r library(ggplot2) data(economics) library(openintro) data("nba_players_19") ``` --- ```r library(openintro) data("nba_players_19") ``` .f3[ The dataset will appear in the global environment, first as an objet of type `promise` and then as a data frame once we evaluate it (use it in any way)] --- # Bonus Explore 1,745 bundled datasets across dozens of packages interactively .b[RDatasets] by Vincent Arel-Bundock https://vincentarelbundock.github.io/Rdatasets/articles/data.html --- class: inverse ## Practice - Load the `fastfood` data bundled with `openintro` - Subset the data to keep only the `restaurant`, `item`, and `cholesterol` columns - Keep only the variables that include information about fat or vitamin content - Which items have values of `calories` > 300 ? - Create a new column that contains the values of `total_fat` multiplied by the square root of `calories` --- class: center, middle, dk-section-title background-image:url("imgs/scl.jpg") background-size: cover # Working `across()` multiple columns --- ## Operating on multiple columns ### .center.b.f2.red[`across()`] .f3[ `filter` or `mutate` multiple columns simultaneously - By position (not recommended), name, or type - Compound selections of columns (e.g. _factor columns that contain "month" in their name_) ] --- ## .b.red[`across()`] arguments: .f2.b.green[`.cols`] Columns to transform .f2.b.pink[`.fns`] Function to apply to each of the selected columns > Expects a function, so function name and arguments can be passed separately --- ```r sneakers %>% mutate(across(c(Price, Rating), `*`, 2)) %>% head(3) ``` ``` ## # A tibble: 3 × 4 ## Name Sport Price Rating ## <chr> <chr> <dbl> <dbl> ## 1 Boost 3000 Running 198 8 ## 2 HiSpeed Running 88 8 ## 3 Super2 Trail 200 10 ``` - `.cols` is a vector with the names of the two columns we're interested in - `*` is the function to apply, `2` is the function argument --- ## Useful column selection helpers .f3[To specify sets of columns, there are a number of useful helper functions that we can use alongside or instead of the `across` function.] .center.f3[ [`dplyr` selection features and helpers]( https://dplyr.tidyverse.org/reference/dplyr_tidy_select.html) ] --- <br/> .ba.ph4[ .b.red[`everything()`] Match all variables ] ```r sneakers %>% mutate(across(everything(), as.character)) %>% head(3) ``` ``` ## # A tibble: 3 × 4 ## Name Sport Price Rating ## <chr> <chr> <chr> <chr> ## 1 Boost 3000 Running 99 4 ## 2 HiSpeed Running 44 4 ## 3 Super2 Trail 100 5 ``` --- .ba.ph4[ .b.red[`!`] Take the complement of a set of variables (negate a selection) ] ```r sneakers %>% select(!c(Sport,Price)) %>% head() ``` ``` ## # A tibble: 6 × 2 ## Name Rating ## <chr> <int> ## 1 Boost 3000 4 ## 2 HiSpeed 4 ## 3 Super2 5 ## 4 Goal 3 ## 5 Goal2.0 4 ## 6 Airball Max 4 ``` --- .ba.ph4.mt1.mb1[ .b.red[`where()`] Selects variables for which a predicate function returns `TRUE` > e.g. subset or transform all numeric variables, determined by `is.numeric()`] ```r sneakers %>% mutate(across(where(is.numeric), log2)) %>% slice(1:3) ``` ``` ## # A tibble: 3 × 4 ## Name Sport Price Rating ## <chr> <chr> <dbl> <dbl> ## 1 Boost 3000 Running 6.63 2 ## 2 HiSpeed Running 5.46 2 ## 3 Super2 Trail 6.64 2.32 ``` --- .ba.ph3[ .b.orange[`matches()`] Match variable names with a regular expression > e.g. drop variables with numbers in their name ] ```r sneakers %>% mutate(across(matches("Rat"), log2)) %>% head ``` ``` ## # A tibble: 6 × 4 ## Name Sport Price Rating ## <chr> <chr> <int> <dbl> ## 1 Boost 3000 Running 99 2 ## 2 HiSpeed Running 44 2 ## 3 Super2 Trail 100 2.32 ## 4 Goal Football 119 1.58 ## 5 Goal2.0 Football 79 2 ## 6 Airball Max Basketball 49 2 ``` --- .ba.ph3[ .center[ .b.red[`:`] Select a range of consecutive variables] ] ```r sneakers %>% select(Name:Price) %>% head ``` ``` ## # A tibble: 6 × 3 ## Name Sport Price ## <chr> <chr> <int> ## 1 Boost 3000 Running 99 ## 2 HiSpeed Running 44 ## 3 Super2 Trail 100 ## 4 Goal Football 119 ## 5 Goal2.0 Football 79 ## 6 Airball Max Basketball 49 ``` --- .ba.ph4[ .b.orange[`-`] Exclude variables (return all elements except the one preceded by the subtraction operator) ] ```r sneakers %>% select(-Sport) ``` ``` ## # A tibble: 11 × 3 ## Name Price Rating ## <chr> <int> <int> ## 1 Boost 3000 99 4 ## 2 HiSpeed 44 4 ## 3 Super2 100 5 ## 4 Goal 119 3 ## 5 Goal2.0 79 4 ## 6 Airball Max 49 4 ## 7 Glide Best 55 5 ## 8 Outdoor King 160 5 ## 9 G90x 120 4 ## 10 Smith Pro 89 4 ## 11 Travel Low 89 3 ``` --- Square root (.red.b[`sqrt()`]) of `Price` and `Rating` ```r sneakers %>% mutate(across(c(Price, Rating), sqrt)) %>% head() ``` ``` ## # A tibble: 6 × 4 ## Name Sport Price Rating ## <chr> <chr> <dbl> <dbl> ## 1 Boost 3000 Running 9.95 2 ## 2 HiSpeed Running 6.63 2 ## 3 Super2 Trail 10 2.24 ## 4 Goal Football 10.9 1.73 ## 5 Goal2.0 Football 8.89 2 ## 6 Airball Max Basketball 7 2 ``` --- ### Square root (.orange.b[`sqrt()`]) of all numeric variables ```r sneakers %>% mutate(across(where(is.numeric), sqrt)) %>% head ``` ``` ## # A tibble: 6 × 4 ## Name Sport Price Rating ## <chr> <chr> <dbl> <dbl> ## 1 Boost 3000 Running 9.95 2 ## 2 HiSpeed Running 6.63 2 ## 3 Super2 Trail 10 2.24 ## 4 Goal Football 10.9 1.73 ## 5 Goal2.0 Football 8.89 2 ## 6 Airball Max Basketball 7 2 ``` --- class: inverse ## Practice - Load the `openintro::fastfood` data - Open the shared document --- class: center, middle, dk-section-title background-image:url("images/trote.jpg") background-size: cover # Pivoting Data --- .pull-left[ ### Wide Data Human-readable Easier entry Easier interactive editing ] .pull.right[ ### Long Data Often meets Tidy Data principles Analysis-ready Maps well to plot axes and aesthetics ]  --- # Reshaping data with 📦 .b.red[`tidyr`] .f3[wide to long: .b.red[`pivot_longer()`]] ```r pivot_longer(data, cols, names_to, values_to) ``` .f3[long to wide: .b.red[`pivot_wider()`]] ```r pivot_longer(data, cols, names_from, values_from) ``` --- .panelset[ .panel[.panel-name[dining] <div id="bsnbzqwqsb" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #bsnbzqwqsb .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #F4F4F9; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #bsnbzqwqsb .gt_heading { background-color: #F4F4F9; text-align: center; border-bottom-color: #F4F4F9; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #bsnbzqwqsb .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #F4F4F9; border-bottom-width: 0; } #bsnbzqwqsb .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #F4F4F9; border-top-width: 0; } #bsnbzqwqsb .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #bsnbzqwqsb .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #bsnbzqwqsb .gt_col_heading { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #bsnbzqwqsb .gt_column_spanner_outer { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #bsnbzqwqsb .gt_column_spanner_outer:first-child { padding-left: 0; } #bsnbzqwqsb .gt_column_spanner_outer:last-child { padding-right: 0; } #bsnbzqwqsb .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #bsnbzqwqsb .gt_group_heading { padding: 8px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #bsnbzqwqsb .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #bsnbzqwqsb .gt_from_md > :first-child { margin-top: 0; } #bsnbzqwqsb .gt_from_md > :last-child { margin-bottom: 0; } #bsnbzqwqsb .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #bsnbzqwqsb .gt_stub { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #bsnbzqwqsb .gt_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #bsnbzqwqsb .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #bsnbzqwqsb .gt_grand_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #bsnbzqwqsb .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #bsnbzqwqsb .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #bsnbzqwqsb .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #bsnbzqwqsb .gt_footnotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #bsnbzqwqsb .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #bsnbzqwqsb .gt_sourcenotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #bsnbzqwqsb .gt_sourcenote { font-size: 90%; padding: 4px; } #bsnbzqwqsb .gt_left { text-align: left; } #bsnbzqwqsb .gt_center { text-align: center; } #bsnbzqwqsb .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #bsnbzqwqsb .gt_font_normal { font-weight: normal; } #bsnbzqwqsb .gt_font_bold { font-weight: bold; } #bsnbzqwqsb .gt_font_italic { font-style: italic; } #bsnbzqwqsb .gt_super { font-size: 65%; } #bsnbzqwqsb .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">area_type</th> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 24px;">district</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 24px;">coffee_shops</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 24px;">fast_food</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 24px;">fish_chips</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 24px;">Residential</td> <td class="gt_row gt_left" style="font-size: 24px;">South</td> <td class="gt_row gt_right" style="font-size: 24px;">2</td> <td class="gt_row gt_right" style="font-size: 24px;">3</td> <td class="gt_row gt_right" style="font-size: 24px;">1</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">Commercial</td> <td class="gt_row gt_left" style="font-size: 24px;">North</td> <td class="gt_row gt_right" style="font-size: 24px;">12</td> <td class="gt_row gt_right" style="font-size: 24px;">6</td> <td class="gt_row gt_right" style="font-size: 24px;">4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">Suburban</td> <td class="gt_row gt_left" style="font-size: 24px;">South</td> <td class="gt_row gt_right" style="font-size: 24px;">2</td> <td class="gt_row gt_right" style="font-size: 24px;">4</td> <td class="gt_row gt_right" style="font-size: 24px;">3</td></tr> <tr><td class="gt_row gt_left" style="font-size: 24px;">Rural</td> <td class="gt_row gt_left" style="font-size: 24px;">East</td> <td class="gt_row gt_right" style="font-size: 24px;">0</td> <td class="gt_row gt_right" style="font-size: 24px;">2</td> <td class="gt_row gt_right" style="font-size: 24px;">0</td></tr> </tbody> </table> </div> ] .panel[.panel-name[Data setup] ```r dining <- tibble::tribble( ~area_type, ~district, ~coffee_shops, ~fast_food, ~fish_chips, "Residential", "South", 2L, 3L, 1L, "Commercial", "North", 12L, 6L, 4L, "Suburban", "South", 2L, 4L, 3L, "Rural", "East", 0L, 2L, 0L ) ``` ] ] .center[Variables (attributes measured):] .fl.w-24[ area type ] .fl.w-25.tc[ district ] .fl.w-25.tc[ type of resturant ] .fl.w-25.tr[ number of restaurants ] --- ## Wide to long ```r dining_long <- dining %>% pivot_longer( cols = c(coffee_shops, fast_food, fish_chips), names_to = "restaurant_type", values_to = "n_restaurants") ``` **`cols`** Which columns have data in their names **`names_to`** Name for the new column to be created from `cols` **`values_to`** Name of the column to be created from the contents of the cells in **`cols`** --- class: middle ```r dining %>% pivot_longer( cols = c(coffee_shops, fast_food, fish_chips), names_to = "restaurant_type", values_to = "n_restaurants") ``` --- ``` ## # A tibble: 12 × 4 ## area_type district restaurant_type n_restaurants ## <chr> <chr> <chr> <int> ## 1 Residential South coffee_shops 2 ## 2 Residential South fast_food 3 ## 3 Residential South fish_chips 1 ## 4 Commercial North coffee_shops 12 ## 5 Commercial North fast_food 6 ## 6 Commercial North fish_chips 4 ## 7 Suburban South coffee_shops 2 ## 8 Suburban South fast_food 4 ## 9 Suburban South fish_chips 3 ## 10 Rural East coffee_shops 0 ## 11 Rural East fast_food 2 ## 12 Rural East fish_chips 0 ``` --- ## Long to wide ```r dining_long %>% pivot_wider(names_from = restaurant_type, values_from = n_restaurants) ``` .b[`names_from`] Which columns' values will be used for new .b[column names] .b[`values_from`] Which column has the .b[cell values] for the new output columns --- ```r dining_long %>% pivot_wider(names_from = restaurant_type, values_from = n_restaurants) ``` ``` ## # A tibble: 4 × 5 ## area_type district coffee_shops fast_food fish_chips ## <chr> <chr> <int> <int> <int> ## 1 Residential South 2 3 1 ## 2 Commercial North 12 6 4 ## 3 Suburban South 2 4 3 ## 4 Rural East 0 2 0 ``` --- class: inverse # Practice - .f3[Load the "`gene_expression.csv`" file] - This table contains made up gene expression values (arbitrary units) for five different fish measured at six time points and sampled from two tissues. - .f3[Pivot the data (wide to long and back to wide)] --- # Other useful functions .f3[**`coalesce()`** - combine data, keeping values from one column and filling any missing data with values from other columns **`fill()`** adjacent values. Helps us avoid unnecessary `NA` values and ambiguity. Similar logic to filling adjacent values in spreadsheets] --- .b.red.f2[`dplyr::coalesce()`] .f3[ - Find and return the first non-NA value at each position for a set of vectors \* _columns in rectangular data are also vectors_ - Returns a single vector with the non-NA values that appear first ] --- .b.red.f2[`dplyr::coalesce()`] ```r x <- c(1, 2, NA, NA, 5) y <- c(NA, NA, 3, 4, 5) z <- c(1, 4, 3, 5, 3) ``` Numeric vectors of the same length - some have missing values. ```r coalesce(x, y, z) ``` ``` ## [1] 1 2 3 4 5 ``` The order of the arguments is important --- ```r x <- c(1, 2, NA, NA, 5) y <- c(NA, NA, 3, 4, 5) z <- c(1, 4, 3, 5, 3) ``` ```r coalesce(z,x,y) ``` ``` ## [1] 1 4 3 5 3 ``` --- .panelset[ .panel[.panel-name[hotels] <div id="tbqfhbtdpk" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #tbqfhbtdpk .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #F4F4F9; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #tbqfhbtdpk .gt_heading { background-color: #F4F4F9; text-align: center; border-bottom-color: #F4F4F9; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #tbqfhbtdpk .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #F4F4F9; border-bottom-width: 0; } #tbqfhbtdpk .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #F4F4F9; border-top-width: 0; } #tbqfhbtdpk .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #tbqfhbtdpk .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #tbqfhbtdpk .gt_col_heading { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #tbqfhbtdpk .gt_column_spanner_outer { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #tbqfhbtdpk .gt_column_spanner_outer:first-child { padding-left: 0; } #tbqfhbtdpk .gt_column_spanner_outer:last-child { padding-right: 0; } #tbqfhbtdpk .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #tbqfhbtdpk .gt_group_heading { padding: 8px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #tbqfhbtdpk .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #tbqfhbtdpk .gt_from_md > :first-child { margin-top: 0; } #tbqfhbtdpk .gt_from_md > :last-child { margin-bottom: 0; } #tbqfhbtdpk .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #tbqfhbtdpk .gt_stub { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #tbqfhbtdpk .gt_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #tbqfhbtdpk .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #tbqfhbtdpk .gt_grand_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #tbqfhbtdpk .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #tbqfhbtdpk .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #tbqfhbtdpk .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #tbqfhbtdpk .gt_footnotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #tbqfhbtdpk .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #tbqfhbtdpk .gt_sourcenotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #tbqfhbtdpk .gt_sourcenote { font-size: 90%; padding: 4px; } #tbqfhbtdpk .gt_left { text-align: left; } #tbqfhbtdpk .gt_center { text-align: center; } #tbqfhbtdpk .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #tbqfhbtdpk .gt_font_normal { font-weight: normal; } #tbqfhbtdpk .gt_font_bold { font-weight: bold; } #tbqfhbtdpk .gt_font_italic { font-style: italic; } #tbqfhbtdpk .gt_super { font-size: 65%; } #tbqfhbtdpk .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 25px;">location</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">guest</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">critic</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 25px;">old</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 25px;">Lisbon</td> <td class="gt_row gt_right" style="font-size: 25px;">NA</td> <td class="gt_row gt_right" style="font-size: 25px;">4.9</td> <td class="gt_row gt_right" style="font-size: 25px;">2.6</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">Porto</td> <td class="gt_row gt_right" style="font-size: 25px;">2.2</td> <td class="gt_row gt_right" style="font-size: 25px;">NA</td> <td class="gt_row gt_right" style="font-size: 25px;">4.3</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">Moura</td> <td class="gt_row gt_right" style="font-size: 25px;">4.5</td> <td class="gt_row gt_right" style="font-size: 25px;">5.0</td> <td class="gt_row gt_right" style="font-size: 25px;">3.4</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">Chaves</td> <td class="gt_row gt_right" style="font-size: 25px;">NA</td> <td class="gt_row gt_right" style="font-size: 25px;">2.7</td> <td class="gt_row gt_right" style="font-size: 25px;">3.5</td></tr> <tr><td class="gt_row gt_left" style="font-size: 25px;">Coimbra</td> <td class="gt_row gt_right" style="font-size: 25px;">2.5</td> <td class="gt_row gt_right" style="font-size: 25px;">NA</td> <td class="gt_row gt_right" style="font-size: 25px;">2.3</td></tr> </tbody> </table> </div> ] .panel[.panel-name[Data setup] ```r hotels <- tibble::tribble( ~location, ~guest, ~critic, ~old, "Lisbon", NA, 4.9, 2.6, "Porto", 2.2, NA, 4.3, "Moura", 4.5, 5, 3.4, "Chaves", NA, 2.7, 3.5, "Coimbra", 2.5, NA, 2.3 ) ``` ] ] --- </br> ```r hotels %>% mutate(new_rating = coalesce(guest, critic, old)) ``` ``` ## # A tibble: 5 × 5 ## location guest critic old new_rating ## <chr> <dbl> <dbl> <dbl> <dbl> ## 1 Lisbon NA 4.9 2.6 4.9 ## 2 Porto 2.2 NA 4.3 2.2 ## 3 Moura 4.5 5 3.4 4.5 ## 4 Chaves NA 2.7 3.5 2.7 ## 5 Coimbra 2.5 NA 2.3 2.5 ``` --- .f2.b.red[`tidyr::fill()`] .fl.w-40[ <div id="rcuvwxrrcd" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;"> <style>html { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif; } #rcuvwxrrcd .gt_table { display: table; border-collapse: collapse; margin-left: auto; margin-right: auto; color: #333333; font-size: 16px; font-weight: normal; font-style: normal; background-color: #F4F4F9; width: auto; border-top-style: solid; border-top-width: 2px; border-top-color: #A8A8A8; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #A8A8A8; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; } #rcuvwxrrcd .gt_heading { background-color: #F4F4F9; text-align: center; border-bottom-color: #F4F4F9; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #rcuvwxrrcd .gt_title { color: #333333; font-size: 125%; font-weight: initial; padding-top: 4px; padding-bottom: 4px; border-bottom-color: #F4F4F9; border-bottom-width: 0; } #rcuvwxrrcd .gt_subtitle { color: #333333; font-size: 85%; font-weight: initial; padding-top: 0; padding-bottom: 6px; border-top-color: #F4F4F9; border-top-width: 0; } #rcuvwxrrcd .gt_bottom_border { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #rcuvwxrrcd .gt_col_headings { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; } #rcuvwxrrcd .gt_col_heading { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 6px; padding-left: 5px; padding-right: 5px; overflow-x: hidden; } #rcuvwxrrcd .gt_column_spanner_outer { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: normal; text-transform: inherit; padding-top: 0; padding-bottom: 0; padding-left: 4px; padding-right: 4px; } #rcuvwxrrcd .gt_column_spanner_outer:first-child { padding-left: 0; } #rcuvwxrrcd .gt_column_spanner_outer:last-child { padding-right: 0; } #rcuvwxrrcd .gt_column_spanner { border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: bottom; padding-top: 5px; padding-bottom: 5px; overflow-x: hidden; display: inline-block; width: 100%; } #rcuvwxrrcd .gt_group_heading { padding: 8px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; } #rcuvwxrrcd .gt_empty_group_heading { padding: 0.5px; color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; vertical-align: middle; } #rcuvwxrrcd .gt_from_md > :first-child { margin-top: 0; } #rcuvwxrrcd .gt_from_md > :last-child { margin-bottom: 0; } #rcuvwxrrcd .gt_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; margin: 10px; border-top-style: solid; border-top-width: 1px; border-top-color: #D3D3D3; border-left-style: none; border-left-width: 1px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 1px; border-right-color: #D3D3D3; vertical-align: middle; overflow-x: hidden; } #rcuvwxrrcd .gt_stub { color: #333333; background-color: #F4F4F9; font-size: 100%; font-weight: initial; text-transform: inherit; border-right-style: solid; border-right-width: 2px; border-right-color: #D3D3D3; padding-left: 12px; } #rcuvwxrrcd .gt_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #rcuvwxrrcd .gt_first_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; } #rcuvwxrrcd .gt_grand_summary_row { color: #333333; background-color: #F4F4F9; text-transform: inherit; padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; } #rcuvwxrrcd .gt_first_grand_summary_row { padding-top: 8px; padding-bottom: 8px; padding-left: 5px; padding-right: 5px; border-top-style: double; border-top-width: 6px; border-top-color: #D3D3D3; } #rcuvwxrrcd .gt_striped { background-color: rgba(128, 128, 128, 0.05); } #rcuvwxrrcd .gt_table_body { border-top-style: solid; border-top-width: 2px; border-top-color: #D3D3D3; border-bottom-style: solid; border-bottom-width: 2px; border-bottom-color: #D3D3D3; } #rcuvwxrrcd .gt_footnotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #rcuvwxrrcd .gt_footnote { margin: 0px; font-size: 90%; padding: 4px; } #rcuvwxrrcd .gt_sourcenotes { color: #333333; background-color: #F4F4F9; border-bottom-style: none; border-bottom-width: 2px; border-bottom-color: #D3D3D3; border-left-style: none; border-left-width: 2px; border-left-color: #D3D3D3; border-right-style: none; border-right-width: 2px; border-right-color: #D3D3D3; } #rcuvwxrrcd .gt_sourcenote { font-size: 90%; padding: 4px; } #rcuvwxrrcd .gt_left { text-align: left; } #rcuvwxrrcd .gt_center { text-align: center; } #rcuvwxrrcd .gt_right { text-align: right; font-variant-numeric: tabular-nums; } #rcuvwxrrcd .gt_font_normal { font-weight: normal; } #rcuvwxrrcd .gt_font_bold { font-weight: bold; } #rcuvwxrrcd .gt_font_italic { font-style: italic; } #rcuvwxrrcd .gt_super { font-size: 65%; } #rcuvwxrrcd .gt_footnote_marks { font-style: italic; font-weight: normal; font-size: 65%; } </style> <table class="gt_table"> <thead class="gt_col_headings"> <tr> <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-size: 21px;">mammals</th> <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" style="font-size: 21px;">biomass</th> </tr> </thead> <tbody class="gt_table_body"> <tr><td class="gt_row gt_left" style="font-size: 21px;">cattle</td> <td class="gt_row gt_right" style="font-size: 21px;">21</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">5</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">7</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">23</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">74</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">wild</td> <td class="gt_row gt_right" style="font-size: 21px;">23</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">65</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">12</td></tr> <tr><td class="gt_row gt_left" style="font-size: 21px;">NA</td> <td class="gt_row gt_right" style="font-size: 21px;">5</td></tr> </tbody> </table> </div> ] .f3[- Fill missing values in a column (top to bottom by default) - Contiguous/adjacent values assumed to be the same - Values only appear when they change along a column - Stops at the next non-missing value] --- .pull-left[ ```r mammal_carbon ``` ``` ## # A tibble: 9 × 2 ## mammals biomass ## <chr> <dbl> ## 1 cattle 21 ## 2 <NA> 5 ## 3 <NA> 7 ## 4 <NA> 23 ## 5 <NA> 74 ## 6 wild 23 ## 7 <NA> 65 ## 8 <NA> 12 ## 9 <NA> 5 ``` ] .pull-right[ ```r mammal_carbon %>% fill(mammals) ``` ``` ## # A tibble: 9 × 2 ## mammals biomass ## <chr> <dbl> ## 1 cattle 21 ## 2 cattle 5 ## 3 cattle 7 ## 4 cattle 23 ## 5 cattle 74 ## 6 wild 23 ## 7 wild 65 ## 8 wild 12 ## 9 wild 5 ``` ]